E cient Estimation of Mutual Information for Strongly Dependent Variables

نویسندگان

  • Shuyang Gao
  • Greg Ver Steeg
  • Aram Galstyan
چکیده

We demonstrate that a popular class of nonparametric mutual information (MI) estimators based on k-nearest-neighbor graphs requires number of samples that scales exponentially with the true MI. Consequently, accurate estimation of MI between two strongly dependent variables is possible only for prohibitively large sample size. This important yet overlooked shortcoming of the existing estimators is due to their implicit reliance on local uniformity of the underlying joint distribution. We introduce a new estimator that is robust to local non-uniformity, works well with limited data, and is able to capture relationship strengths over many orders of magnitude. We demonstrate the superior performance of the proposed estimator on both synthetic and real-world data.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Efficient Estimation of Mutual Information for Strongly Dependent Variables

We demonstrate that a popular class of nonparametric mutual information (MI) estimators based on k-nearest-neighbor graphs requires number of samples that scales exponentially with the true MI. Consequently, accurate estimation of MI between two strongly dependent variables is possible only for prohibitively large sample size. This important yet overlooked shortcoming of the existing estimators...

متن کامل

Wavelet Based Estimation of the Derivatives of a Density for m-Dependent Random Variables

Here, we propose a method of estimation of the derivatives of probability density based wavelets methods for a sequence of m−dependent random variables with a common one-dimensional probability density function and obtain an upper bound on Lp-losses for the such estimators.

متن کامل

Mine: Mutual Information Neural Estimation

We argue that the estimation of the mutual information between high dimensional continuous random variables is achievable by gradient descent over neural networks. This paper presents a Mutual Information Neural Estimator (MINE) that is linearly scalable in dimensionality as well as in sample size. MINE is backpropable and we prove that it is strongly consistent. We illustrate a handful of appl...

متن کامل

Hydrograph Estimation based on Various Components of Rainfall Using Adaptive Neuro-Fuzzy Inference System in Kasilian Watershed

Flood hydrograph preparation and estimation are considered a comprehensive information for soil and water managers and planners. While it is not simply possible preparing it for all watersheds. Therfore suitable flood hydrograph estimation and modeling seems to be necessary using available rainfall data. The study area is located in Kasilian representative watershed in Mazandaran province compr...

متن کامل

Estimation of the Survival Function for Negatively Dependent Random Variables

Let be a stationary sequence of pair wise negative quadrant dependent random variables with survival function {,1}nXn?F(x)=P[X>x]. The empirical survival function ()nFx based on 12,,...,nXXX is proposed as an estimator for ()nFx. Strong consistency and point wise as well as uniform of ()nFx are discussed

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2015